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ABSTRACT 

In an era of spiraling costs, librarians must 
carefully assess literature use among their patron groups when making 
decisions on selection, deselection, and storage of journals. A 
citation analysis was conducted using a group of 52 papers submitted 
by research physicians at the Summa Health System (Akron, Ohio) for a 
research award competition during the years from 1993 to 1995. The 
study investigates the use ratio of journals to books, the age of the 
journal articles, the country of publication and the language of the 
cited journal articles, and the citation patterns of the residents in 
regard to use of general journals, specialty journals in their own 
field, or specialty journals in other fields. Out of 869 citations, 
91.3% of them referred to journal articles, with 252 separate journal 
titles represented. While only 43.1% of the journal articles were 
published since 1990, 69% of the books were in this category. Most of 
the journals, 82.1%, were published in the United States and 11.3% 
were published in England. All but two articles were written in 
English. Geieral medical journals were cited 11.7% of the time, 
specialty journals in the resident’s own field 31.9% of the time, and 
journals in other specialties 54.5% of the time. Of the cited 
journals, 30% of them appeared on consulted lists of core medical 
subscription recommendations, and 43% of them were carried by the 
Summa libraries. Six tables accompany the data, and four appendices 
provide lists of cited journals and citation count worksheets. 
(Contains 55 references.) (BEW) 
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ABSTRACT 



In this time of spiraling costs, it is particularly important for 
librarians to carefully assess the literature uses and needs of their 
particular patron groups when making decisions on selection, 
deselection and storage of journals. Several types of use studies have 
been conducted in libraries to aid in these decisions. 

A citation analysis was conducted using a group of fifty-two papers 
submitted by resident physicians at Summa Health System for a 
research award paper competition for the years 1993-1995. The study 
was designed to determine the use ratio of journals to books, the age of 
the literature cited, the country of publication and language of the cited 
journal articles, and the citation patterns of residents in regard to their 
use of general medical journals, specialty journals in their own field of 
medicine, and specialty journals in other medical fields. 

869 citations, with 91 .3% of them referring to journal 
articles and 6.7% to books. A total of 252 journal titles were cited. The 

journal articles cited tended to be older than the cited books. While 
43. 1% of the journal articles were five years old or newer, 69% of the 
books were in this category. Most of the journals cited, 82 1% were 
published in the United States, with 1 1.3% published in England. All 
but two of the cited articles were written in English. General medical 
journals were cited in 1 1 .7% of references. The authors of the papers 
cited journals in their own specialty in 31 .9% of the citations, and in 
other specialties in 54.5% of the citations. 

The list of journals cited by this group of resident physicians was 
compared to two well-known lists of core medical journals for small and 
medium-sized medical Ubraries, and to the current subscription list of 
the Summa libraries. The Abridged Index Medicus List of Journals 
Indexed and the Brandon/Hill Hst each include about 30% of the cited 
tities, while 43% of the titles are on the current subscription list of the 



highlights the value of local studies, and the need to 
study the literature used for a variety of purposes by the many patron 
groups served by a library. ^ ^ 



Master’s Research Paper by 
Roberta Shorter 



Adviser 



O 

ERIC 



B.A., University of Minnesota, 1965 
M.L.S., Kent State University, 1995 



Approved by 






Date 



ro - 



ii 



4 



TABLE OF CONTENTS 



LIST OF ILLUSTRATIONS 

I. INTRODUCTION 

Purpose of the Study 

Hypothesis 

Definition of Terms 

II. REVIEW OF THE LITERATURE 

Use Studies in Libraries 

Citation Analysis 

Bibliometric Laws 

Sources of Citations for Analysis 

Citation Analysis in Academic Libraries 

Citation Analysis Using Journals or Indexes 

Use Patterns of Journals in Medical Libraries . . . 

Core Lists 

Use Studies in Medical Libraries 

Assumptions and Limitations of Citation Analysis 
Uses of Citation Analysis 

III. METHODOLOGY 

rV. ANALYSIS OF DATA 

Source Types of Citations 

Age of Citations 




iii 






V 

1 

5 

5 

5 

7 

7 

9 

10 
11 
12 
14 

14 

15 
17 
19 
21 
22 
24 

24 

25 



Journals Cited 

Countiy of Publication 

Language of Journal Articles 

Specialty of Journals Cited 

Comparison of Journal Titles Avith Core Lists and 
Subscriptions 

V. SUMMARY AND CONCLUSIONS 

APPEND;[X 

A. Source Types of Citations 

B. Journals Cited; Frequency, Percentages, Core Lists 

and Current Subscriptions 

C. Country of Publication of Cited Journals 

D. Coding Sheet 

REFERENCE LIST 



27 

28 
29 

29 

30 



32 



36 

37 

45 

46 

47 




iv 



b 















LIST OF ILLUSTRATIONS 



Figure Page 

1. Source type of cited works 25 

2. Age of cited books 26 

3. Age of cited journal articles 27 

4. Country of publication of cited Journals 28 

5. Specialty of journals cited: general medical journals, 

specialty journals in the same specialty as that of the 

author, and journals in specialties other than that 

of the author 3 q 

6. Percent of cited journal titles on core lists and subscription 



31 



I. INTRODUCTION 

One of the purposes of a hospital medical library is to provide a 
collection of books, journals and other reference sources which will meet 
the basic inforaiation needs of the various groups which make up the 
library's patrons. Ever-expanding networks and consortia, in 
combination with modem technology, have made it possible to locate and 
borrow books from other libraries or obtain photocopies or faxes of 
articles through interlibraiy loan. A limited number of journals are 
available online in full-text versions, but the much discussed "library 
without walls" is not yet a reality, and is not likely to be in the near 
future. Until such time as a multitude of problems have been solved 
involving copyright, ownership of information, and. more basically, who 
wiU fund and manage "instant access," it is stiU important that the local 
hospital or medical center libraiy make readily available at least a core 
collection of the most frequently needed materials for libraiy patrons. 

The spiraling prices of library materials, and particularly of journal 
subscriptions, have strained libraiy budgets and forced Ubrarians to look 
critically at subscription lists to determine which journals are used often 
enough by the library's vailous patron groups to justify subscriptions 
and which are needed so infrequenUy that they may be more 
economically obtained by other means. Libraiy space is also at a 
premium and expanding periodical collections can quickly fill available 
shelves. Decisions need to made about the value of keeping long runs of 
journals in the libraiy as opposed to disposing of older journals or 
sending them to remote storage areas. 

Lists of standard or c ire journals are available to help medical 
librarians select those that will be most useful for their patrons. Two of 
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the most commonly consulted are the Brandon/HiU Selected List of 
Books and Journals for the Small Medical Ubrary (Brandon and Hill 
1995) and the Abridged Index Medicus List of Journals (National 

Library of Medicine 1 994) . 

The Brandon/HlU list, according to its authors, has been used 
since its inception in 1965 as a tool for developing or updating libraries 
in hospitals. The primaiy focus is patient care, rather than research. 
Basic science journals are not included on the list, on the theory that 
their low use would not justify their cost, and needed artiicles could be 
more economically obtained elsewhere (Brandon and Hill 1995). 

The introduction to the Index Medicus list describes AIM [Abridged 
Index Medicus) as a subset of the National Library of Medicine’s 
MEDLARS database of medical literature. AIM contains citations to 
biomedical literature judged to be of immediate interest to practicing 
physicians, covering all clinical fields (National Library of Medicine 
1994). A help screen for MEDLINE, the NLM online database, suggests 
that limiting a search to AIM journals "is a good way to limit retrieval to 
journals commonly found in hospital and medical library collections" 
(National Library of Medicine 1995). 

As useful as lists of standard and core journals are to medical 
librarians in selecting journals for their libraries, addiUonal information 
is needed. Judgments about which new joumaJs to buy, which current 
subscriptions to continue, where to keep older journals, or whether to 
keep them at all, are best made when there is accurate information 
available about the various groups the library serves and the nature of 
their particular needs and patterns of use. 
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Several types of studies are employed in libraries to measure the 
use of resources. Circulation statistics are compiled to see which 
materials or which types of materials are checked out. Interlibraiy loan 
requests may be tabulated and analyzed to learn what types of materials, 
and, in the case of journals, which titles are borrowed the most 
frequently. In-house use of library materials may also be measured. 
Libraries which provide photocop3dng service can count photocopy 
requests. In reshelving studies, patrons are asked not to reshelve items 
they use in the library, and a count is made by library stalf before books 
and journals are reshelved. Patron self-reporting studies ask patrons to 
fill out a questionnaire or initial or check a slip of paper attached to 
unbound or bound journals to indicate their use of an item. Library staff 
may use direct observation techmques and record their impressions of 
materials used and the types of use. 

Another technique used to measure journal use, the one used in 
this stuvdy, is citation analysis. Indexes, subject bibliographies, or core 
journals or texts in a field may be analyzed to see which sources are 
cited. The citation indexes, and particularly their machine-readable 
electronic databases, may be searched to learn which journals or authors 
have been cited. These products. Science Citation Index, Social Science 
Citation Index, and Arts and Humanities Citation Index, all created by 
the Institute for Scientific Information, make it possible to search huge 
databases of citation information and to analyze complex relationships 
between citing and cited papers. 

Local use studies may also use citation analysis. The papers of a 
particular group of researchers or students may be obtained, and the 
citations in tliem analyzed to determine the formats or titles most often 
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used. Local use citation studies reported in the library literature have 
most often involved researchers in university libraries. One reason for 
the concentration of citation studies in academic libraries may be the 
high volume of publication by university researchers, and also the fact 
that their publications are likely to be covered in the standard indexing 
tools. The faculty status of many librarians in academic settings, the 
requirement that they conduct and publish research, and the time 
allowed for them to pursue these activities probably also contribute to 
the volume of research involving academic libraries. 

There are fewer reports on the use of literature in medical libraries. 
Most ot them involve reshelving or user seli-reporting studies. The stall's 
of larger hospitals, particularly teaching hospitals with residency 
programs, usually include researchers who have published papers, but 
citation analyses of their publications have not been reported. This 
author found one master’s research paper which analyzed citations in 
the publications of researchers at a medical school and compared them 
to the library's holdings (Hanrahan 1994). 

The literature use of hospital residents as a group appears not to 
have been studied by citation analysis. A major reason for this is 
probably that hospital residents usually do not publish papers in 
significant numbers, nor are other papers they have written readily 
available, and it is therefore difficult to obtain citations to analyze. An 
active research program at Summa Health System, however, supports 
and encourages residents in research projects. Papers are submitted for 
an annual research award paper competition and are coUected by the 
Research Department. 
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Purpose of the Study 

The purpose of this study was to determine various characteristics 
of the literature used by one group of hospital library patrons, residents, 
for one particular purpose, the writing of research papers for a 
competition. 



Hypothesis 

It was h)^p)othesized that the study would show that these 
residents used a high percentage of journals as opposed to books, in 
accordance with the usual use patterns of researchers in the hard 
sciences. It was also expected that a relatively small number of Journal 
titles would account for a large percentage of the citations with the 
remaining citations scattered over a larger number of titles, and that a 
higher percentage of citations would be to more recent literature, 
probably within the previous five years. A fairly high degree of conformity 
to the journal titles in the AIM and Brandon/Hill lists was also predicted. 

It was hoped that this research would provide useful information 
about this facet of library use that could be combined with other kinds of 
library use data to aid in the determination of a core group of journals 
tailored to the needs of this particular library community. 

Definition of Terms 

The following defimtions were used in this study; 

Citation: A bibliographic reference to a work. Items in footnotes, 
endnotes, bibliographies and reference lists were included. 

Works referred to in the text of a paper but not included in one of 
the above formats were not counted. Each cited title or article was 
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counted only once per paper. References made as "Op. cit." 

"Il>id" were not counted. 

Book: Textbooks, handbooks and other individual monographs which 
are not part of a series were included here. 

Journal: Serial publications which are printed on a regular basis, such 
as monthly, quarterly, or annually. TTie terms journal and 
periodical are used interchangeably. Hardbound annual 
publications which are issued as a series, for example Annual 
Review of Medicine are also included here. 

Conference report: An individually published report of a conference or 
meeting. Reports of meetings which were published as an article 
in a journal are included with journals. 

General Medical Journal, specialty medical journal: Subject categories of 
journals in Index Medicus List of Journals Indexed were used. 

For subject identiil cation of journals not indexed in Index Medicus, 
Medical and Health Care Books and Serials in Print was used. 



II. REVIEW OF THE LITERATURE 



In the current climate of static or reduced library funding and 
spiraling costs, librarians need data on the information use of various 
patron groups in order to wisely apportion limited library acquisition 
budgets. This need for reliable data regarding the use of various types of 
information sources, and particularly the use of serials by students, 
faculty and researchers in academic and special libraries, has prompted 
numerous studies anti papers on ways to measure patron use of these 
resources. 

Studies have shown that researchers in different fields use 
Uterature differently. Devin reviewed more than fifty studies on the 
monograph/serial citation ratios for researchers in different subject areas 
and found that researchers in the hard sciences use a higher percentage 
of serials than those in the humanities and social sciences. The 
percentage of serials used was 19.9% for researchers in English 
literature, 27.1% for historians, 38.8% for sociologists, 85.2% for 
researchers in medicine and 93.6% for chemists. Devin suggests that 
librarians should consider the appropriate monograph/serial ratio for 
each discipline represented in the coUection to determine the appropriate 
ratios of expenditures (Devin 1989). 

Use Studies in Libraries 

There are many ways to conduct use studies in libraries. 

Circulation statistics may be compiled quite easily, especially with 
automated circulation systems, but serials do not circulate in many 
libraries, so other methods are needed to determine usage of serials. 
Common techniques include gatliering statistics on photocopying or 
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interlibrary loan requests, counting periodicals I’eshelved by library staff, 
collecting user self-reporting data and direct observation of library 
patrons. 

A self-reporting study done in the Science and Engineering Library 
at the Universiiy of Buffalo compared results with a reshelving study 
done four years earlier. The self-reporting study indicated a 40% drop in 
usage of current periodicals which were reported as heavily used in the 
earlier study. The researcher suggests possible explanations for this 
apparent drop in use, and p>oints out advantages and disadvantages of 
the two methods. The self-reporting method is relatively inexpensive to 
cany out. Forms are attached to journals, and users are a.sked to initial 
the form with each use. fhere are, however, disadvantages which can 
lead to inaccurate results. Users may be tempted to manipulate the data 
by over-reporting usage of favorite journals. Others may ignore the 
study. Any level of use, from picking up a periodical and scanning the 
table of contents to reading several articles has the same weight. 
Reshelving studies are more expensive to conduct, and require more staff 
time because volumes must be reshelved frequently to avoid uncounted 
multiple uses. The technique is less vulnerable to deliberate 
manipulation by users, but users sometimes reshelve journals, leading to 
under-reporting (Naylor 1994). 

Direct observation is another way of measuring periodical use. 
Researchers in the Evans library at Texas A & M University saw tills 
method as a way of avoiding the dependence on user participation and 
cooperation involved in user self-reporting studies, and on staff efficiency 
in reshelving studies. While direct observation is better suited to 
identifying types of use (whether a journal is scanned or actually read. 
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for example), problems arise with observational errors, the difficulty in 
collecting data regarding specific tides, high costs for staff time, and the 
possible interference effects of this more intrusive method (Bustion, 
Eltinge and Harer 1992). 

Citation Analysis 

Various types of citation analysis, the method employed in this 
research study, are also used to measure serial use. Citation analysis, 
which was used as early as 1917 as "statistical bibliography" 
(Subramanyam 1980), involves collecting and analyzing data on 
references from indexes and subject bibliographies, footnotes, reference 
lists and bibliographies of published or unpublished works. These 
studies are used in many ways. Libraiy serials collections are developed 
or evaluated through the use of core or ranked lists of most-used 
journals. Sociological and historical analyses are made of the use of 
Uterature in various fields. Journals in a field are ranked on the basis of 
their importance or impact. For example, one researcher analyzed library 
science journals to determine which journals were the most prestigious 
for publications by health sciences librarians, and would therefore be of 
the most value to those seeking tenure and promotion (Fang 1989). 

The influence or prestige of individual scholars or institutions may 
be affected by where they publish or are cited. Controversy can arise, 
especially when ranking on this basis is done formally, as in the 
publication Science Watch (Taubes 1993). 

Citation studies may range from simple counts of citations for a 
given document or set of documents, to "impact rankings" of journals 
which allow comparisons of journals publishing widely var 3 dng numbers 
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of articles. Linda Smith describes these methods, and also discusses 
more complex techniques Involving "bibUographic coupling" (where the 
reference lists of two documents cite the same document or documents), 
and co-citation analysis" (where two documents are jointly cited in the 
same subsequent document) (Linda Smith 1981). 

Science Citation Index created by the Institute for Scientific 
Information in 1964. followed by Social Science Citation Index and Arts 
and Humanities Citation Index, provide an immense database of citation 
data. In machine-readable form, these databases allow for the speedy 
collection and manipulation of data and have increased the ease and 
popularity of citation studies. 

Bibiiometric Laws 

Some patterns or principles have been observed in citation 
analysis. Subramanyam. in a discussion of the history of citation 
analysis, discusses Bradford's law of scattering which describes a pattern 
in which a small core of journals in any field contains a large proportion 
of the significant articles, and the rest of the articles are scattered 
throughout a larger number of journals (Subramanyam 1980). This 
phenomenon is also known as dispersion or productivity (Wallace 1987). 
This concept has also been referred to as the "80/20" rule, which states 

that 20% of the collection meets 80% of user needs (Burnham. Shearer 
andWaU 1992). 

Another phenomenon often studied in citation analysis is the aging 
or obsolescence of the literature. A very high percentage of use is 
reported for recent articles, while use of older items drops off quickly 
(Wallace 1987). Other authors report that citations to a particular 
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document wUl peak in the second or third year after publication. 
Similarly, the largest number of references made in particular sources 
are to documents published two or three years earlier (White and McCain 
1989). Researchers have also found that the literature of different fields 
ages or becomes obsolete at different rates, with earlier obsolescence for 
articles in medical and chemistry journals, for example, compared to that 
for articles in social science journals (Glanzel and Schoepflin 1995). 

Sources of Citations for Analysis 

Citations for analysis may be obtained from several sources. There 
are advantages and disadvantages for each type of source, and tlie 
resulting data can be used in different ways. Secondary sources such as 
indexing and abstracting journals are easily accessible, and counting 
their citations is a fairly mechanical process, but indexing services vary 
in their coverage, both of journals and over time. Some journals may be 
covered completely and others selectively. There is also usually a 
language or country bias. National bibliographies give a total picture of 
the monographs of a country, but there is no selectivity or assessment of 
quality, and serials are not covered. Subject bibliographies usually 
indicate a measure of quality, but go out of date quickly and are 
generally restricted to monographs (Brittain and Line 1973). 

Primary publications and the citations in them are the most 
common source for citation studies. Advantages of using primary 
sources are that the most cited items may be assumed to be those most 
wanted by library users. There is also some implicit indication of quality. 
Analyses may be made by subject and language, and obsolescence 
patterns may be observed. Primary sources also have disadvantages. 



Researchers may not cite all of the sources they use, or conversely, use 
all of the sources they cite. Authors publishing in a journal are more 
likely to cite other articles from that same journal rather than those from 
other journals. In addition, journals which are readily available to a 
writer are more likely to be cited than those which require more time and 
effort to obtain. Researchers should be aware of the particular 
limitations of various types of citation data and choose sources 
appropriate for their needs (Brittain and Line 1973). 

Citation Analysis In Academic Libraries 

Many studies involving citation analysis have been reported ii. the 
libraiy literature, but few of them have dealt with medical literature. A 
majority of citation analyses involve local use in academic or research 
libraries, and generally usage by faculty or other researchers. 

Studies in the applied physical sciences at the Georgia Institute of 
Technology (Dykeman 1994) and in forestry at the University of Florida, 
Gainesville (Haas and Lee 1991) analyzed citations in the publications of 
faculty and compared them to the institutions' libraiy holdings. A study 
ot citations by faculty in the Physics Department at Emory University 
emphasizes a technique in which citations were downloaded from 
SciSearch on DIALOG, edited using a word processing program and 
analyzed using a spreadsheet program (Greene 1993). A study of 
journal use by the Biology Department at Temple University analyzed 
citation data from faculty pubUcations, doctoral dissertations and 
preliminary doctoral qualifying briefs. This study points up the 
importance of studying use patterns of different patron groups to get a 
more complete profile of journal use (McCain and Bobick 1981). 

10 



12 



o 

ERIC 



Citations made in publications of researchers at Battelle, an 
international contract research and development organization, are 
analyzed in a master’s research paper. Core lists of Journals for the 
organization and for the disciplines represented are presented, and the 
unique problems of collection development for a special library which 
serves researchers in a non-university organization are discussed (Blum 
1993). 

Some citation analyses have been conducted in academic settings 
using citations from student papers. A study at a Texas university 
analyzed journal use by graduate students in psychology, using the 
citations in their theses and dissertations (Sylvia and Lesher 1995). 

There has been less attention in the literature to sources used by 
undergraduate students. One study used papers from two universities 
and two liberal arts colleges and compared the format and age of 
materials used by students in different disciplines and at different 
undergraduate levels (Magrill and St. Clair 1990, St. Clair and Magrill 
1992). Papers by senior psychology students at a large university and a 
small liberal arts college were analyzed in another study which found 
that about twenty journals accounted for 80% of the citations. The 
authors also found a wide diversity in the remaining titles cited at the 
two institutions, and tliey suggest that only local use studies are of 
practical value to libraries (Hardesty and Oltmanns 1989). 

A study funded by the United States Office of Education measured 
information use by almost 2000 high school students in fifteen high 
schools. The report of this study is geared toward school Ubraiy media 
specialists and presents survey data on the types of libraries used by 
these students and a citation analysis of the format and age of the 
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literature they cited in their papers. This report gives detailed 
information on the methodology of the study and includes copies of the 
questionnaire used and sample coding sheets for the citation analysis 
(Mancall and Drott 1983). 

Citation Analysis Using Journals or Indexes 

Some citation studies analyze citations in the standard or 
significant journals or indexes in a discipline to ci eate core lists of 
journals or determine which journals have the greatest impact. Two 
studies in social work use this technique. One analyzes the citations in 
two national social work journals and the Encyclopedia of Social Work 
and compares this data with the citations by faculty in a school of social 
work (Wiberly 1982). A similar study uses citations from a current 
awareness bulletin and compares them to a local use study fWilson 
1994). A study at Purdue University compared citations in three 
interdisciplinaiy core history journals with the history coUection in the 
Purdue University libraries (Henibel 1990). Political science coUections 
m five Washington, D. C. area university libraries are evaluated using 
citations from a prestigious politicsd science review journal and five 
political science journals (Nisonger 1983). 

Use Patterns of Journals In Medical Libraries 

This research project uses citation analysis to study the use 
patterns of medical journals by one group of hospital library patrons. 
This kind of information is of particular interest and importance to 
health sciences librarians, who need to know which resources are the 
most valuable or will receive the most use in their libraries. 
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It has been shown that researchers in medicine use a higher 
percentage of serials as opposed to monographs (Devin 1989). One 
study analyzed citations from ten highly ranked internal medicine 
journals and citations in two British and five American internal medicine 
textbooks. Data was also tabulated on the local use of monographs or 
serials by physicians, residents and students to answer questions or 
topics from hospital rounds. The researchers weighted the averages of 
the citations in the serials and textbooks to allow for different numbers of 
articles published, and found a ratio of 88% serial citations to 12% 
monograph citations. The ratio for serials to monographs used for the 
questions from hospital rounds was 89.5% serials to 10.5% monographs 
(Burdick. Butler and Sullivan 1993). 

Core Lists 

Standard core lists of recommended medical literature are 
available and are often consulted by health sciences librarians in 
developing and evaluating their collections. Two of the best known 
among medical librarians are the Brandon/ Hill list, which covers 
medical, nursing and allied health monographs and journals, and the 
Abridged Index Medians list of indexed periodicals. Core lists contain 
selected titles which are central to a field, and therefore most likely to be 
used. One researcher discusses and compares five core lists of medical 
journals - the Brandon/Hill list, the Moll list, the Abridged Index Medians 
list, the AUyn list and the Bell list. The merged lists contain 450 titles. 
Thirty-seven titles were on all five lists, and thirty-four more titles were 
on four of the lists. The AIM ]isi had the greatest agreement with the 
consensus and the smallest number of unique titles (Usdin 1979). 
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Journal Citation Reports, which is produced by the Institute for 
Scientific Information as a supplement to Science Citation Index, ranks 
journals by an "impact factor." ISI's impact factors are designed to allow 
valid comparisons of Journals which publish different types and volumes 
of articles. An "immediacy index" records how quickly an article is cited 
after it is published. Garfield provides lists of general and internal 
medicine journals ranked by citations and by impact factor, and also a 
list of biomedical research journals ranked by impact factor (Garfield 
1986). 

Usdin matched the data from her core list comparisons to rankings 
from Journal Citation Reports and found results that were complex and 
difficult to explain. She recommends that Journal Citation Reports be 
used only in conjunction with other selection tools (Usdin 1979). 

Analyses of indexes have been used in the medical field, as in other 
disciplines, to develop lists of core journals. This technique is 
particularly useftd in fields which draw on a variety of disciplines. One 
study uses three indexes in the field of rehabilitation to develop a core 
list of journals (Bohannon and Roberts 1991). Another examines serial 
use in citations from journals abstracted in Behavioral Medicine 
Abstracts. The ninety-one resulting journal titles are compared with the 
Brandon/Hill list and the Abridged Index Medicus list. While 38% of the 
titles appeared on at least one of the lists, fifty-six heavily used titles do 
not appear on either list, an indication of the multidisciplinary nature of 
the field (Slater and Slater 1994). 

Some studies of medical literature use emphasize the application of 
new technologies to the techniques of citation analysis. An analysis of 
citations on the topic "gait" detscribes in detail a technique involving CD- 
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ROM databases and a reprint management program to study key 
journals and databases (Burnham. Shearer and Wall 1992). 

Use Studies in Medical Libraries 

In addition to using the standard core lists to measure use of 
materials in medical libraries, researchers have designed local use 
studies of various types. Circulation statistics provide data on the use of 
items which circulate, but serials often do not circulate. Studies of 
interlibraiy loan requests can show items not owned by the Ubraiy which 
are being used by libraiy patrons (Lacroix 1994). Local use studies of 
medical literature have been published in which data was gathered from 
reshelving counts, user self-reporting surveys and analyses of sources 
used to answer medical information questions. Most local use studies 
reported in the literature have used these techniques, rather than 
citation analysis. 

A study at Himmelfarb Health Sciences Library at George 
Washington University used barcodes on all bound and unbound 
journals added to the library collection to measure in-house use. Before 
journals were reshelved, the barcodes were scanned into a computer and 
the resulting data was used to help make decisions on journal 
cancellations required by a strained budget. The researchers emphasize 
that other factors were taken into consideration including the frequency 
of publication, available indexing, and the library's holdings and binding 
patterns (Bader and Thompson 1989). 

A reshelving study at the University of Minnesota Bio-Medical 
Library used samples from two different time periods. Lists of top ranked 
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journals used in the library are presented, along with data on the age of 
the journals used (Tibbetts 1974). 

Another study analyzed data collected from reshelving counts, 
interlibr-ary loan requests, storage requests and journal reserves. The 
cost and space required for each journal (density of use) and other 
information on categories of use. use by different patron groups, year of 
publication, and subject was added. The objective of this compUcated 
study was to learn the content, cost and space requirements of a 

collection that would fill aU the demands of the library's users (Bastille 
and Mankin 1980). 

A brief report on a study in the Brockton Hospital Ubraiy. 
Brockton. Massachusetts uses statistics on the age of journals used for 
photocopy requests, in-house reading and circulation of periodicals to 
evaluate policies for the retention of journals. Varied use patterns for 
different journals were reported by the author of the article, who 
recommends that decisions on length of retention of journals be made on 
an individual basis, rather than by a blanket policy (Kamenoff 1977). A 
similar study in a pediatric teaching hospital used circulation data, 
reshelving statistics and an analysis of interlibrary loan requests to 
measure journal use (Joan Smith 1970). 

A study measuring a different type of use analyzed journals used 
by medical librarians to answer 144 patient-care related questions in the 
Library of Medicine at the Medical College of Peimsylvania. The 191 titles 
used to answer the questions were matched against the Brandon/Hill 
list. The researchers found that 30% of them were Brandon/Hill titles. A 
total of 904 articles were used to answer the questions. Only eighteen 
titles furnished 47.6% of the articles. Almost two-thirds of the articles 
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used had been published in the last five years, and most of the articles 
used that were older than five years came firom core Journals (Miller and 
Moore 1984). 

Only one study was found which analyzed citations in papers of 
local medical researchers. A study of the references cited in publications 
by researchers at the Northeastern Ohio Universities College of Medicine 
is reported in a master's research paper. The researcher compared the 
journals cited to the Ubraiy's holdings. Citations were collected from a 
"hand count" and from a DIALOG search in SciSearch. An important 
finding is that there are significant problems with tJie accuracy of citation 
data, with the coverage and accuracy of SciSearch, and with the citation 
analysis process. Readers are advised against using citation data alone 
to decide on the purchase or cancellation of journals (Hanrahan 1994). 

Assumptions and Limitations of Citation Analysis 

Researchers who interpret citation analysis data need to be aware 
of the assumptions that are implicit in this technique. Citation analysis 
assumes that authors refer to all the sources they used (or at least the 
most important ones), that they used all the documents they cite, that 
citing a document implies that it has some quality or merit, that authors 
cite the best works on their subjects, that there is a semantic 
relationship between a citing document and the cited document, and 
that all citations are equal (Linda Smith 1981). CitaUon studies are 

limited by the extent to which these assumptions are accurate for a given 
document. 

Possible inaccuracies in data can involve differences in ciUng 
patterns in various fields, wide variations in citation counts from one 
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year to anotiier, and errors t'. die citations included. Other inaccuracies 
result from the characteristics of citation Indexing tools such as Science 
Citation Index. There are problems with selective indexing, with 
authority control of names, auid with indexing when there are multiple 
authors (Linda Smith 1981). 

Other researchers also highlight limitations of citation analysis and 
the cautions that should be observed in using it. Some writers point out 
that citations reflect only one type of use, and that other uses which are 
not reflected in citations need to be considered. Journals which are 
written and read for research purposes are more likely to be cited than 
those that focus on reviews, news or current awareness (Garfield 1972). 
Journals of an applied nature are cited much less frequently than 
research oriented journals (Scales 1976). Citation analysis often reflects 
the use of sophisticated scholars and their use should not be assumed to 
be the same as the use of undergraduates or other library patrons 
(Broadus 1977). Literature other than serials, such as textbooks, 
encyclopedias and other reference works are often heavily used, but not 
cited (Subramanyam 1980). Citation data may also be biased by what 
is available in a local collection. Articles cited may those that were 
readily available, rather than the best resources for the topic (Todorov 
1988). Finally, caution is suggested in comparing literature from 
different disciplines, because citation practices vary, as does the rate of 
obsolescence in different fields (Stankus and Rice 1982), and small fields 
within a discipline are often under-represented in citation analyses 
(Archibald and Ftnifter 1987). 



Uses of Citation Analysis 

Citation analysis and other types of use studies are often employed 
by librarians, particularly in academic libraries, in conjunction with the 
standard core lists, to develop and evaluate loced collections, "deselect" 
journal titles which are not used enough to justify their cost, and consign 
seldom-used materials to remote storage. 

Researchers suggest the use of citation analysis or other local use 
studies to tailor serials collections to the needs of individual libraries 
(Greene 1993). Some recommend the use of Journal Citation Reports 
combined with a study of local use Information and user surveys to aid 
in deselection of journals (Dombrowskl 1988. Thomas Smith 1985). 
Others stress the value of local rather than generalized studies, along 
with the need to look at other factors such as uses per subscription cost 
and uses per foot of shelf space (Line and Sandison 1975. Line 1978). 

Studies of the cited use oi journal literature by local patron groups 
hi libraries, when applied with caution and an awareness of the 
assumptions and limitations of the technique, can be a useful part of an 
on-going evaluation of the needs of libraiy users. 
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The source of citations used in this study was papers submitted to 
the annual Research Award Paper Competition by residents in thirteen 
accredited residency training programs at Summa Health System, an 81 1 
bed. two-hospital health care facility in Akron. Ohio. All papers 
submitted in 1993. 1994 and 1995 were analyzed. Two papers, reports 
of experiments which contained no citations, were omitted. The 
remaining fifty-two papers contained 874 citations. Of these citations, 
only five were so incomplete that it was impossible to determine what 
source was being cited. These five citations were omitted, leaving a total 
of 869 citations for analysis. 

The citations from the papers were analyzed to determine the 
format of the literature used (book, joumsd. conference report, etc.) and 
the age of the cited publication. Journal articles were further analyzed 
as to journal title, the frequency of citation to each journal title, the 
country of publication, language of the article, and whether the article 
was in a general medical journal or in a specialty journal, either in the 
field of the paper's author, or in another specialty field. This information 
was coded and entered into EDO. a data entry and modification system 
at Kent State University. The data was analyzed using SAS (Statistical 
Analysis System). All percentages were rounded to one decimal place. 
Appendix D is a copy of tlie coding sheet used to enter the data for each 
citation. Information on the age of the cited sources was coded in 
discrete categories to allow for easier analysis. 

The list of journals cited by this group of residents was compared 
to the Abridged Index Medlcus Ust of Journals Indexed and the 
Brandon/Hill list to see how these residents' use of literature correlates 
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with two lists of standard or core journals. Finally, the current journal 
subscription list of the two Summa libraries was checked to learn which 
of the cited journals were currently received. 



IV. ANALYSIS OF DATA 



The papers were written by residents in eleven of the thirteen 
accredited residency programs. There were many more papers in some 
specialties than others. Eleven of the fifty-two papers were written by 
residents tn orthopedics, while there was only one paper each in 
ophthalmology and urology. Because of this extreme variation in the 
number of papers in different disciplines, no attempt was made to 
compare the citing patterns of residents in the various specialties. 

The fiffy-two papers analyzed contained a total of 869 usable 
citations. The number of citations per paper ranged from a low of three 
in one paper to a high of fifty-one in another paper. 



Source Types of Citations 

The citations were divided into seven source types: books, 
conference papers, unpublished works, personal communications, 
government documents, journal articles and "other." As expected, a vast 
majority of the citations, 793 or 91.3%, were to journal articles, a 
pattern consistent with the typical use of literature types in the hard 
sciences. This figure is comparable to the 85.2% reported by Devin for 
use of serial Uteratm*e by medical researchers (Devin 1989) and the 88% 
and 89.5% reported by Burdick ((Burdick, Butler and Sullivan 1993). 
Books were cited in fifty-eight or 6.7% of the 869 references. The 
remaining eighteen citations were to sources of other types. Figure 1 
shows this breakdown of source types. For a complete tabulation of 
citations to the seven types of sources, see Appendix A. 
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Other Sources 2% Books 6. 7% 




Journal Articles 91 .3% 
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Figure 1 . Source type of cited works 

Age of Citations 

The citations in the papers were also analyzed to determine the age 
of the sources used. It was expected that a high proportion of sources 
cited would be five years old or less. In fact, citations for the first five 
years totaled only 45.6% of the total, with 67.7% of the citations being to 
sources ten years old or less. Sources older than twenty years accounted 
for 14.4% of the total. Among the oldest articles cited were one from 
1902 and two from 1912. 

An interesting pattern emerged when the age of citations to books 
and to journal articles were looked at separately. Of the fifty-eight books 
cited in the papers, 69% were five years old or newer, while only 5.2% 
were over twenty years old. (See Figure 2) The data for citations to the 
793 journal articles is shown in Figure 3. When journals were cited. 
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43.1% of the articles were five years old or newer, with 15.5% of the 
articles older than twenty years. 

An explanation for this difference in citing patterns for books and 
Journals may be that small and medium-sized libraries are far more likely 
to have older volumes of journals on the shelves than they are to have 
older editions of books. An older ediUon of a book is often discarded or 
put in storage when a new edition is added to the collection. In 
addition, perhaps older journal articles are tradiUonally seen in the hard 
sciences as more legitimate sources for historical background and 
literature review than are older books. 
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5 Years 1 5.5% 



4 Years 12.1% 



3 Years 12.1% 

2 Years 15.5% 




6 to 10 Years 12.1% 

1 1 to 1 5 Years 6.9% 

16 to 20 Years 6.9% 



Older Than 20 Years 5.2% 
1 Year 1 3.8% 



Figure 2. Age of cited books 
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6 to 10 Years 23.1% 



11 to 15 Years 11.2% 




Figure 3. Age of cited journal articles 



Journals Cited 

A total of 252 different journal titles were cited in these papers. 
Citation frequency ranged from forty citations for Clinical Orthopaedics 
and Related Research, to one citation each for 129 journal titles. The top 
three journals, all in orthopedics, had over 10% of the total citations, a 
reflection of the high participation by orthopedic residents in the 
research paper competition. (Papers by orthopedic residents accounted 
for 25.4% of the total citations in the flfty-two papers). Appendix B 
includes a list of all the journals cited and the frequency and percentage 
of citations. 

Bradford's law of scattering would predict that 20% of the tides, or 
about fifty journals on the fist, should account for 80% of the citadons. 

In fact, the top 20% of the cited Journals account for only about 59% of 
the citadons. Citadons to nearly 43% of the journal ddes must be 
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counted to reach the 80% level. At this point, more than half the journal 
titles which received only two citations are included. These figures 
indicate use of a much longer and more varied list of journals by these 
residents than might be predicted by bibliometric laws. 

Country of Publication 

The country of publication was also noted for each journal citation. 
Over 82% of the journals cited are published in the United States. An 
additional 1 1.3% are published in England, with the remaining 6.6% 
published in other countries. Figure 4 shows this distribution of 
countries of publication, with the complete tabulation by country of 
publication appearing in Appendix C. 



Other 6.6% England 11.3% 




United States 82.1% 
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Figure 4. Country of publication of cited journals 
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Language of Journal Articles 

All but two of the 793 journal articles were written in the English 
language, with one article in German and einother in Portuguese. 
Journals were cited that are published in several countries where 
English is not the primary’' language, but most of these journals are 
published in English, or have an English language edition. Foreign 
language journals indexed in Index Medicus or Medline have an English 
language translation of the title, and usually an abstract in English. It is 
possible for researchers to use only the abstract to get some information 
from the indexed article, without having to read the original article. It is 
not known whether the entire article or just the abstract was used for the 
two cited foreign language articles. 

Specialty of Journals Cited 

The final characteristic of the citations which was studied was the 
specialty of the journals cited in relation to the specialty of the citing 
author. Figure 5 shows these relationships. General medical journals 
accounted for ninety- three or 11.7% of the journal article citations. The 
authors of the papers cited journals in their own specialty in 31.9% of 
the journal citations, and in other specialties in 54.5% of the citations. 
The "other" category includes scientific journals which are not specifically 
medical journals. 
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other 1.9% 



General Medical Journal 11.7% 



Specialty, Different 54.5% 




Specialty, Same 31.9% 



Figures. Specialty of journals cited: general medical journals, 

specialty journals in the same specialty as that of the author, and 
journals in specialties other than that of the author 

Comparison of Journal Titles Cited with Core Lists and Subscriptions 

The final element of analysis of the journal citations in these 
papers was a comparison of the journal titles cited with the Abridged 
Index Medicus List of Journals Indexed, the Brandon/Hill list, and the 
current subscription list of the Summa libraries. The thirteen journal 
titles with the highest number of citations, a total of 221 citations or 
almost 28% of the total citations in the papers, were listed on both core 
lists and also on the current subscription lists of the libraries. The 
percentage of cited titles that are on each of the core lists and on the 
subscription list is represented in Figure 6. Of the total list of 252 
journal titles, seventy-three or 29% of them are on the AIM Mst. The 
Brandon/Hill Ust includes seventy-six or 30% of the 252 cited titles. The 
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current subscription list of the two libraries includes 109 or 43% of the 
cited titles. 



50.0 r 
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Figure 6. Percent of cited journal titles on core lists and 
subscription list 







31 

38 

BEST COP Y AVAILABL E 



V. SUMMARY AND CONCLUSIONS 



This study presents a picture of the use of medical and scientific 
literature by a group of residents at Summa ifealth System who entered 
a research paper competition. The study showed a high use of journal 
articles as opposed to books and other source types, a finding that was 
expected in the medical field and consistent with other research. 

The literature used by the residents was older than might have 
been expected given the usual bibliometric predictions on the rapid aging 
and obsolescence of literature in the hard sciences. Most papers, 
however, involved a review of the fiterature on the author's topic, leading 
to the citation of a large number of older articles. 

There were 252 journal titles cited in the papers. The expected 
clustering of 80% of the citations in 20% of the articles did not occur. 
Citations were spread over a larger number of journal titles, representing 
a greater scattering of citations than predicted by bibliometric laws. 

Because participation in the research paper competition is 
voluntaiy. the papers studied represent a self-selected sample of 
residents who chose to enter the competition. The number of residents 
in each residency program is different, with some programs being larger 
than others. The participation level in this competition by residents in 
some programs is also higher than in others. The study particularly 
reflects the strong research orientation of the Orthopedics Department, 
which recently opened a new musculoskeletal research laboratory. It is 
assumed that this emphasis on orthopedic research had an effect on the 
number of orthopedic papers and the number of citations to orthopedic 
journals. No comparisons were made of the use of literature by residents 
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tn different residency training programs because the sample is not 
equally representative of all departments. 

The coimtry of publication of the cited journals reveals a very high 
use of journals pubUshed in the United States, and even when foreign 
journals are used, the articles are almost exclusively in the English 
language. Medicine tn the United States is strongly oriented toward the 
western world, and this orientation is reflected in the indexing sources 
commonly used and the literature which is readily available. 

While no comparative statistics were attempted because of the 
disproportionate number of papers and citations in the various 
specialties, it is interesting to note that authors in some specialties c ited 
journals within their own specialty more than other authors did. 
Orthopedic residents tended to cite orthopedic literature almost 
exclusively, while residents in Emergency Medicine, Internal Medicine 
and Family Practice used the literature of other specialties more often. 
ITiis is to be expected, given the multidisciplinaiy nature of the later 
specialties. There are also many more orthopedic journals published and 
indexed than, for example, internal medicine journals. The 1994 edition 
of List of Journals Indexed in Index Medicus Usts forty-one journals in 
Orthopedics with seventeen of them published in the United States. 
Twenty-one Internal Medicine journals are listed, with only four of them 
published in the United States. 

The comparison of the cited journals with the core lists shows a 
strong agreement between the yl/A/and Brandon/Hlll lists. There were 
only seventeen differences between the two Usts on the entire 252 journal 
tlUes. ^i/l/Usted 29% of the journal titles, while the Brandon/HiU Ust 
contained 30% of the titles. 
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The higher percentage of cited journal titles that are on the current 
subscription list of the Summa Libraries may be partly due to the 
tendency of library patrons to use literature which is easily available, but 
it also reflects the attention that has been paid in collection development 
to the needs and specialties of the particular patron groups served by the 
two llbimies. 

It is important to note that these papers represent one type of 
literature use - the writing of research papers by a self-selected group of 
authors. The sources cited for this purpose may very well be different 
from those which are used by the same residents for current awareness 
or clinical/patient care purposes. Their use of literature is also not 
generalizable to all residents at Summa. Residents who did not enter 
this competition are not represented at all by this data. They may veiy 
well use other sources for their research -oriented activities, current 
awareness and clinical information needs. Likewise, these results are 
not generalizable to literature use by residents at other institutions. 

Finally, residents are only one part of the diverse health care 
community served by the Summa Libraries. Researchers, hospital-based 
and attending physicians, hospital administrators, faculty and 
administrators in medical and nursing education programs, nursing 
students, medical students, staff and students in a variety of laboratory 
and allied health Adds, and other hospital employees are also served by 
the libraries. In addition, circuit library service is provided for a number 
of small hospitals and agencies in the area. The information needs and 
the literature used by this diverse group of library patrons have not been 
represented in this study. 
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This study of the sources used by residents in these papers, 
therefore, provides part of a picture of the information sources used by 
the hospital community. In order to use such information to make 
decisions on journal selection or deselection, it would be necessary to 
gather similar information for other groups served by the hospital 
libraries. Further research with a variety of types of use studies are 
needed to more completely describe literature use by the diiferent patron 
groups for their diverse information needs. 

This study highlights the importance of analyzing local use of 
literature sources, rather than rel)dng solely on core lists and information 
derived from studies in other libraries. E)ach medical library serves a 
unique institution with its own programs, projects, and areas of 
specialization. The more information that can be gathered about the 
information needs of that institution and its diverse group of library 
patrons, the better able the libraiy will be to provide efficient, cost- 
effective and high quality library service. 
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Appendix A 



Source Types of Citations 



Source Type 


Frequency 


Percent 


Book 


58 


6.67 


Conference Paper 


4 


0.46 


Unpublished Work 


4 


0.46 


Personal Communication 


1 


0.12 


Government Document 


8 


0.92 


Journal article 


793 


91.25 


Other* 


1 


0.12 



* This was a citation to a computer program. 
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Appendix C 



Country of Publication of Cited Journals 



Country 



Frequency 



Percent 



Australia i 

Belgium 2 0.3 
Brazil i q[i 
Canada 5 0.6 
China i o!l 
Denmark n 2 4 
England 90 11 ! 3 
Finland l ^ 0.1 
France 1 o!l 
Germany 8 1*0 
Ireland 2 ols 
Italy 5 o!e 
Japan 2 03 
Netherlands 2 0.*3 
Norway 2 ols 
Scotland 1 0*1 
Sweden 4 0 ! 5 
Switzerland 2 0.3 
Turkey i o!l 
United States 651 821 
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Appendix D 
Ccxling Sheet 



1. Paper # (2 digits) 

2. Citation # (2 digits) 

3. Source type 

JL Book (including handbooks) 

_2_ Encyclopedia, dlctionaiy 

_2_ Conference paper, paper presented at meeting 

Jl Unpublished work 

_5_ Personal communication 

_6_ Government document 

Journal (including annual reviews, annuals and proceedings that are 
published monthly, quarterly or annually as serials) 

_S_ Other 

4. Age of cited work (see separate sheet with date ranges given for each of the three 

years of papers) 

_L Year of paper (year of paper and previous calendar year) 

J2l- Within 2nd calendar year before paper 
_3_ Within 3rd calendar year before paper 
Within 4th calendar year before paper 
^ Within 5th calendar year before paper 
JL Within 6th to 10th calendar year before paper 
_Z Within 1 1th to 15th calendar year before paper 
_S_ Within 1 6th to 20th calendar year before paper 
_Q_ 21st year before paper or older 

If source type is journal (7). continue. Otherwise stop here. 

5. Journal title (abbreviation from Index Medicus or other standardized format) 

6. Country of publication 



Qi Australia 
02 Belgium 
OS Canada 

04 China 

05 Denmark 
QS_ England 


07 Finland 

08 France 
OS Germany 
IQ Ireland 
li Italy 

12. Japan 


12. Netherlands 
14, Norway 
15 Scotland 
IS Sweden 
IZ Swifebrland 
12 Turkey 
12 United States 
22 Brazil 


7. Language 






_L English 
.2. German 


J2. Chinese 
A. French 


Italian 
_£ Japanese 
_Z Portugese 


8. Journal subject 







_1_ Medicine (general medical journals are grouped here) 
_2- Specialty , same as author of citing autltor 
^ Specialty, other than specially of citing author 
Jl Other 
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